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p ersonal identification and handwri t j n g identification 
by means of biometric data 



The invention relates to a biometric acoustic writing 
system, in which the sound signals which are produced 
by the hand-guided movement of the pen are recorded 
acoustically by means of a microphone which is 
integrated in a pen appliance, and they are then 
processed to form biometric data. The invention also 
relates to a method for personal identification and 
personal verification by means of a biometric writing 
system, and to a method for handwriting identification 
and/or handwritten sketch identification. 

Currently, there are a large number of competing 
biometric systems. The majority of the biometric 
systems are currently in the form of systems which use 
finger lines as identification features, followed by 
systems for face identification and so-called Iris 
scanners . 

The importance of electronic password and signature 
identification is becoming continuously "more important 
as a result of increasing electronic networks in all 
fields of business life. Conventional contracts are 
closed by the two contract partners by providing a 
signature for documentation. The signature is in this 
case used for verification purposes and is specific to 
the person providing the signature. When contracts are 
closed in the field of E -commerce, the ' contract 
partners have to provide an electronic signature for 
their identification, its validity must be verified. 
One precondition for the widespread use of an 
electronic signature for verification of the identity 
of the contract partners is a high degree of confidence 
for the electronic signature that is provided cannot be 
falsified. 
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Furthermore, the signature which is provided must be 
identified with a high identification rate, and must be 
transmitted to a remote computer. It is also essential 
that the electronic signature for a large group of 
people can be entered in a simple manner without any- 
special knowledge . 

The writing systems which are already known do not 
satisfy these requirements, and in some cases have 
serious disadvantages . 

Pens with acceleration sensors are known (for example 
acceleroraeter-based pens from British 

Telecommunications) for detection of acceleration 
forces while writing with the pen. The disadvantage of 
this known pen is the comparatively low identification 
rate for signature verification, because the data area 
is only two-dimensional. Furthermore, the pen must 
always be aligned in the same way while writing, that 
is to say the angle between the pen and the substrate 
must be very largely constant, so that a natural 
writing behaviour is not ensured when providing the 
signature. Furthermore, an acceptable identification 
rate is dependent on a minimum writing speed since, 
otherwise, the measured acceleration forces are too 
small . 

A large number of writing systems identify handwritten 
text or characters by means of the writing substrate. 
All such writing systems have the disadvantage that 
specific writing substrates are required, with 
appropriate signal processing. 

US 4,513,437 discloses a data input pen for signature 
verification. The pen contains a pen for visualization 
of what has been written on a substrate, as well as 
piezoelectric pressure and acceleration sensors for 
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recording acceleration forces which are caused by 
transversal writing movements of the pen on the writing 
substrate. Furthermore, the pressure on the pen is 
recorded in a direction parallel to the axis of the 
5 pen . 

The disadvantage of the data input pen described in 
US 4,513,437 is that pronounced acceleration forces are 
required during writing. Major direction changes and/or 
10 speed changes in straight-line writing movements are 
thus required for signal recording. 

The documents US 5 294 792' A, US 5 333 209 A, 
US 6 208 755 Bl, DE 196 44 109 Al and DE 100 28 138 Al 
15 describe writing systems for recording writing pressure 
dynamics by means of pressure sensors and writing 
movement data by means of CCD sensor systems. 

Writing systems are known in which a pen for carrying 
20 out hand-guided movements on a substrate is integrated 
in a pen appliance. The pen appliance contains an 
optical sensor device for recording the movements 
carried out with the pen, as movement or position data. 
Furthermore, a further pressure sensor device is 
25 integrated in the pen appliance, in order to record the 
forces which occur while the movements of the pen are 
being carried out. A data processing unit is used to 
calculate biometric data as a function of the recorded 
movement data and the recorded writing pressure data. 

30 

Commercial writing appliances already exist which are 
equipped with the sensor techniques mentioned above. 

For example, the so-called Anoto-digital pen describes 
35 a writing system in which written text can be 
transferred to a computer. The pen is in the form of a 
ball-point pen with an integrated digital camera. A 
force sensor detects that the tip of the refill has 
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been placed on a special paper. The special paper is a 
paper with a printed pattern of dots. The integrated 
digital camera of the Anoto pen perceives the writing 
movements of the pen over the pattern of dots. The 
camera records a large number of images per second. 
Each recorded image is compared with the subsequent 
image, and an integrated processor uses them to 
calculate the direction change and the movement path of 
the pen refill, which is integrated in the pen, taking 
account of how the pen is held. A large number of dots 
are printed on the special Anoto paper, which are 
appliefd using carbon-based ink and are separated from 
one another by less than 1 mm. Each Anoto special paper 
has a different pattern of dots, so that the individual 
papers can be distinguished from one another. A light- 
emitting diode (LED) illuminates the ink dots, with 
contain carbon, absorb longwave infrared light, and can 
thus be identified by the integrated digital camera 
within the Anoto pen. The Anoto special paper allows 
the relative movement of the pen to be determined, as 
well as the absolute position of the pen on the paper. 

The disadvantage of the Anoto pen is that special paper 
is required and the way in which the pen is held is 
predetermined. Furthermore, the confidence against 
falsification of the signature verification is too low, 
since no personal-specific biometric data is 
determined. The Anoto pen does not record the 
individual writing pressure on the tip of the pen 
refill. 

In addition to electronic signature verification, there 
is an increasing need for electronic recording of 
handwritten texts and characters. In conventional 
methods for handwriting identification, hand-guided 
writing movements are recorded optically, and the 
written image data that is determined is then converted 
by means of OCR methods to ASCI coded data. A number of 
TRIl\605845vl 



possible word candidates are conventionally selected 
during the process of identification of a written word. 
The number of possible word candidates in this case 
becomes greater the less clear the handwriting is, and 
the greater the number of similar words that exist. The 
most probable word candidate from the various possible 
word candidates is then determined and emitted, by 
means of statistical evaluation. The statistical 
evaluation is carried out,, as in the case of automatic 
speech identification, by means of so-called word 
sequence statistic programs, which calculate the 
statistical probability of the occurrence of the 
respective word candidate on the basis of the previous 
words which have been identified. The most probable 
word candidate that occurs is selected, and is then 
emitted. The disadvantage of such conventional methods 
for handwriting identification is that the number of 
possible word candidates is in many cases very high, so 
that the identification rate is often very low. 

Writing systems for signature verification, for 
personal identification or verification and for 
handwriting and speech identification have to have a 
very high degree of confidence against falsification, 
and have to have low error rates. Furthermore, the 
technical implementation must not be excessively 
complex, in order to allow a wide field of application. 

The object of the present invention is thus to provide 
a multifunctional biometric acoustic writing system, 
which has low error rates and offers a very high degree 
of confidence against falsification, allows acceptance 
that is as wide as possible, and can be produced very 
cost-effectively . 

According to the invention, this object is achieved by 
a biometric writing system having the features 
specified in Patent Claim 1. 
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The biometric writing system according to the invention 
is preferably used as a multifunctional input system 
for personal identification in biometry. 

5 

The writing system according to the invention can 
replace or add to the tablet of PC or a virtual 
desktop, using normal writing paper as the input 
medium. The writing system according to the invention 
10 can also be used in life sciences for diagnosis and 
therapy, in medicine and in psychology. 

The invention relates to a biometric acoustic writing 
system having a pen housing for carrying out hand- 

15 guided movements on a substrate, at least one 
microphone, which is integrated in the pen housing, for 
acoustic recording of sound signals which are caused by 
the hand-guided movements; and a data processing unit' 
for calculation of biometric data as a function of the 

20 recorded sound signals. 

The biometric, acoustic writing system according to the 
invention allows the generation of personal-specific 
biometric reference data which is unique for that 
25 person. This makes it possible to verify the identity 
of someone who is writing, and even to identify them by 
comparison with stored reference data. 

The biometric writing system according to the invention 
30 also allows handwriting identification and handwritten- 
sketch identification in order to determine and emit 
handwritten characters in digital form. 

The biometric writing system according to the invention 
35 preferably includes a data processing unit for 
reconstruction of handwritten characters and texts from 
the recorded sound signals. 
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A pen is preferably provided in the pen housing, which 
has been placed on the substrate and is guided on the 
substrate . 

5 This offers the particular advantage that the person 
has a natural sense of writing, since the biometric 
writing system according to the invention behaves like 
a normal pen appliance with a pen refill filled with 
ink . 

10 

The acoustic writing noise which is caused by the 
friction of the pen on the substrate during the hand- 
guided writing movement is preferably transmitted as a 
structure-borne sound signal via the pen and/or 
15 additionally as an airborne sound signal via the 
surrounding air, through the microphone which is 
integrated in the pen housing. 

Since the writing noise can be transmitted not only via 
20 the surrounding air but also a structure-borne sound 
signal, the writing system according to the invention 
is also suitable for fields of application in 
acoustically loud areas. 

25 The microphone is preferably mechanically coupled to 
the pen in order to transmit the structure-borne sound 
signal . 

In this case, in one particular preferred embodiment, 
30 the microphone is mechanically coupled to a sound body, 
which is connected to the pen, in order to transmit the 
structure-borne sound signal. 

The sound body is preferably in the form of a resonator 
35 for specific natural frequencies. 

In one preferred embodiment of the biometric writing 
system according to the invention, the microphone is 
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arranged in an airborne sound chamber which is provided 
in the pen housing. 

The airborne sound chamber is preferably in the form of 
5 a resonator for specific natural frequencies. 

Together with the resonators, the microphone is 
preferably surrounded by sound insulation, which is 
intended to attenuate environmental noise and passes 
10 sound signals only via the pen. 

The airborne sound chamber can be coupled to the 
surrounding air via a housing opening which is provided 
in the pen housing. 

15 

The housing opening can be closed by means of a 
mechanical closure device in order to suppress external 
noise. 

20 This offers the user the capability to suppress 
interference noises from the environment by closing the 
opening . 

When the housing opening is open, the microphone 
25 preferably acoustically records the internal and 
external writing noise which is caused by the hand- 
guided writing movement as a structure-borne and 
airborne sound signal and/or acoustically records a 
speech signal which originates from a person. 

30 

The sound body for the external writing noise is 
preferably formed by the writing substrate. 

When the housing opening is open, the microphone can 
35 optionally be replaced by a second microphone, which is 
not acoustically isolated, in the pen housing. 

The acoustic recording of the speech signal in addition 
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to the acoustic recording of the writing noise allows 
the error rate for identification of that person to be 
considerably reduced. The combination of the evaluation 
of the speech signal and of the writing noise results 
in the biometric writing signal having a very high 
degree of identification confidence. 

The microphone preferably converts the recorded 
acoustic sound signals to an electrical sound signal. 

The electrical sound signal is preferably converted by 
an analogue/digital converter to sound signal data for 
digital data processing by means of the data processing 
unit . 

In this case, the writing noise and the speech signal 
are recorded simultaneously or successively by means of 
the microphone. 

The sound signal data which is produced can preferably 
be stored in a memory unit for the biometric writing 
system . 

The writing substrate is preferably composed of any 
desired paper. 

In this case, the writing substrate is preferably a 
fixed substrate which has a specific pronounced surface 
roughness and hardness. 

In one particularly preferred embodiment of the 
biometric, acoustic writing system, a loudspeaker is 
provided in the pen housing in order to reproduce 
recorded microphone system, in order to reproduce 
stored biometric reference data, and in order to 
reproduce spoken information. 

Preferably and in addition, a speech signal which 
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originates from the person is recorded acoustically, 
and corresponding sound signal data is produced for 
speaker identification or speech identification. 

5 The writing noise itself can likewise be evaluated 
using the algorithms and methods for speaker and speech 
identification . 

. In a further preferred embodiment of the biometric, 
10 acoustic writing system or acoustomechanical writing 
system according to the invention, a second sensor 
device is also provided, which records the writing 
dynamics via the static and dynamic pressure, and 
records the writing speed via the oscillation of the 
15 pen, of the hand-guided writing appliance which has 
been placed on the substrate. 

In a further preferred embodiment of the biometric, 
acoustic writing system or acoustomechanical writing 
20 system according to the invention, at least three 
sensors are provided for simultaneous recording of the 
writing forces which occur and of oscillations of the 
pen refill in three spatial directions. 

25 In a further preferred embodiment of the biometric, 
acoustic writing system or acoustomechanical writing 
system according to the invention, an inclination 
sensor device is also provided, which determines the 
inclination of the hand-guided writing appliance which 

30 has been placed on the substrate, and thus 
predominantly records the finger motor movements while 
writing . 

In one preferred embodiment of the biometret writing 
35 system an optical sensor device is also provided, which 
records position data of the hand-guided pen movement 
via image signals from the surface of the writing 
substrate . 
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In a further preferred embodiment of the biometric 
writing system, a pressure sensor device is also 
provided, which records the static and dynamic pressure 
of the hand-guided pen which has been placed on the 
substrate . 

The microphone is preferably an electric microphone, a 
piezoelectric microphone, a piecoresistive microphone 
or capacitive microphone. 

An interchangeable pen refill is preferably provided in 
the pen of the biometric writing system. 

This pen refill can preferably be extended from the pen 
by means of a switching device. 

In one preferred embodiment of the biometric writing 
system according to the invention, the optical sensor 
device has imaging optics in order to image the 
substrate surface, and has a converter device for 
conversion of the optical imaging signal to an 
electrical signal . 

The imaging optics are in this case preferably in the 
form of optical lenses and/or glass fibres. 

The converter device is preferably a CCD camera or a 
photodiode array. 

Furthermore, a diode light source is preferably 
provided in order to illuminate the substrate surface. 

In one particularly preferred embodiment, a laser diode 

is also provided in order to illuminate the substrate 

surface and an optical grating which is integrated in 
the pen. 
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The optical sensor device preferably records the 
movements which are carried out by the pen by 
comparison of the image sequences, recorded with the 
time offset, of the substrate surface. 

5 

In one alternative embodiment, the optical sensor 
device records the movements which are carried out by 
the pen by comparison of the image sequences, which are 
recorded with the time offset, of the interference of 
10 the laser light which is reflected on the grating and 
on the substrate surface. 

In one preferred embodiment, when it is in an extended 
state on the substrate, the pen refill emits a writing 

15 liquid while the hand-guided movements are being 
carried out, which writing liquid increases the 
structuring of the substrate surface, such that the 
optical recording of the hand-guided movements by means 
of the optical sensor device is simplified and a 

20 natural writing style is produced. 

The data processing unit is preferably integrated in 
the pen housing or in an external receiving unit. 

25 The external receiving unit is in this case preferably 
a computer, a microcontroller, a mobile telephone, a 
credit card reader, a fax machine or a printer. 

The recorded sound signal data, optical movement data 
30 and recorded multidimensional pressure data is 
preferably transmitted via a data transmission path 
from the pen housing to a data processing unit which is 
integrated in an external receiving unit. 

35 The biometric writing system according to the invention 
preferably has a scrambling unit, which is contained in 
the pen housing, for scrambling sound signal data, 
optical movement data and pressure data. 
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The data transmission path in this case preferably uses 
wires, or is wire-free. 

5 The biometric writing system according to the invention 
preferably has a data memory for storage of biometric 
reference data, position data for the writing movement, 
and spoken information. 

10 The biometric reference data is preferably calculated 
by the data processing unit from the sound signal data 
which is recorded while writing and speaking a word, 
from optical movement data, from mechanical pressure 
data and is stored in a reference data memory. 

15 

The reference data memory is preferably a microchip in 
an identification card or a credit card, is an 
authorization magnetic card, or is a memory unit for a 
computer or for the biometric writing system. 

20 

The written and spoken words and characters are 
preferably pin codes, passwords, names or texts. 

The position data for the writing movement is 
25 preferably calculated from the sound data signals which 
are recorded while writing a word, from optical 
movement' data and from mechanical pressure data, and is 
stored in the data memory for handwriting 
identification . 

30 

The reference data memory is in this case preferably 
integrated in the pen housing. 

The data processing unit which is integrated in the 
35 external receiving unit is preferably connected to a 
reference data memory. 



In one preferred embodiment of the biometric writing 
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system, the external receiving unit has a reading unit 
for reading portable biometric reference data. 



The external receiving unit is preferably connected to 
5 a reference data memory via a data network to a 
database . 

The data network is preferably the Internet. 

10 In one preferred embodiment of the biometric writing 
system, the data processing unit compares the 
calculated current biometric data with the stored 
biometric reference data for its verification and 
identification . 

15 

If the current biometric data largely matches the 
stored reference data, the data processing unit 
produces an identification and/or verification 
indication signal . 

20 

In one particularly preferred embodiment of the 
biometric writing system according to the invention, if 
the current biometric data completely matches the 
stored biometric reference data, the data processing 
25 unit identifies the current biometric data as a stolen 
copy of the stored reference data, and produces a 
corresponding warning signal. 

The data processing unit produces a discrepancy 
30 indication signal in the event of a discrepancy between 
the current biometric data and the stored biometric 
reference data. 

In one preferred embodiment of the biometric writing 
35 system, this writing system has at least one actuator 
which is operated after production of the 
identification and/or verification indicator signal. 
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The invention also provides a method for generation of 
personal-specific biometric reference data, having the 
following steps : 

acoustic recording of hand-guided writing movements 
5 which are carried out by a person using a pen on a 
substrate while writing a character, a word or a word 
sequence, and production of corresponding sound signal 
data ; 

storage of the sound signal data that is produced, as a 

10 digital sound time signal; 

calculation of associated frequency spectra as a 
spectrogram from the time-segmented sound signal data 
by means of a fast Fourier transformation; 
determination of amplitude time signals of selected 

15 frequencies in order to record the amplitude dynamics 
in the calculated spectrogram; 

calculation of an associated frequency spectrum from 
the selected amplitude time signals by means of fast 
Fourier transformation; 
20 determination of first biometric data from the sound 
and oscillation intensity of the digital time signals 
by means of feature extraction; 

determination of second biometric data from the 
calculated spectrogram of the time-segmented sound and 
25 oscillation time signals by means of feature 
extraction; 

determination of third current biometric data by means 
of feature extraction from the calculated frequency 
spectrum of the amplitude time signals; 
30 determination of fourth current biometric data by means 
of feature extraction from recorded dynamic writing 
pressure, oscillation and inclination data. 

In one preferred embodiment of the method, the method 
35 steps are carried out more than once in order to 
generate personal-specific biometric reference data, 
and the respectively determined reference data is 
evaluated statistically, with the evaluated data being 
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scrambled as personal-specific reference data, and then 
being stored. 

In one preferred embodiment, the statistically 
5 evaluated, personal-specific reference data is stored 
in a microchip in an identification card, in a 
microchip in a credit card, in an authorization 
magnetic card or in a memory unit for a computer or for 
a writing system. 

10 

Preferably and additionally, a speech signal which 
originates from that person is acoustically recorded, 
and corresponding sound signal data is produced. 

15 Furthermore, in one preferred embodiment of the method 
according to the invention, the writing forces which 
occur in at least one spatial direction while the 
writing movement of the pen is being carried out is 
recorded by means of pressure sensors in order to 

20 generate personal-specific biometric reference data, 
and at least one-dimensional writing pressure data is 
produced in a corresponding manner. 



Furthermore, the writing movements which are hand- 
25 guided on the substrate while writing a character, a 
word or a character/word sequence with the pen are 
preferably recorded optically, and corresponding 
writing movement data is produced. 



30 The biometric reference data is preferably determined 
from the sound intensity of the digital time sound 
signal by means of feature extraction, from the 
calculated frequency spectrum of the sound time signal 
by means of feature extraction, and from the optical 

35 movement data and writing pressure data produced while 
writing and speaking a word, and is stored in the 
reference data memory in a scrambled form as personal- 
specific biometric reference data. 
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Data reduction of the static and dynamic writing 
movement data, of the writing pressure data, of the 
optical movement data and of the sound signal data is 
5 preferably carried out by means of cluster formation. 

Feature parameters are preferably extracted from the 
data-reduced cluster data. 

10 The generated biometric reference data is preferably 
stored inside or outside the pen. 

The invention furthermore provides a method for 
verification of a person having the following steps: 
15 acoustic recording of hand-guided writing movements 
which are carried out by a person using a pen on a 
substrate while writing a character, a word or a word 
sequence, and production of corresponding sound signal 
data; 

20 production of writing pressure signal data for the 
writing pressure and production of oscillation signal 
data for oscillations which are transmitted from the 
pen to at least one pressure and oscillation sensor; 
storage of the signal data that is produced at digital 

25 time signals; 

calculation of frequency spectra as a spectrogram from 
the stored time-segmented sound and oscillation time 
signals by means of fast Fourier transformation; 
determination of amplitude time signals of selected 

30 frequencies in order to record the amplitude dynamics 
in the spectrogram of the sound and oscillation time 
signals; calculation of an associated frequency 
spectrum from the selected amplitude time signals by 
means of a fast Fourier transformation. 

35 

In this case, a speech signal which originates from the 
person is preferably additionally acoustically 
recorded, and corresponding sound signal data is 
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In this case, preferably and additionally, a speech 
signal which originates from the person is acoustically 
5 recorded, and corresponding sound signal data is 
produced . 

A speech signal which originates from the person is 
preferably additionally acoustically recorded, and 
10 corresponding sound signal data is produced. 

The writing movements which are hand-guided on the 
substrate while writing or sketching a character, a 
word or a character/word sequence with the pen are 
15 preferably additionally optically recorded, and 
corresponding position data is produced. 

Preferably and additionally, the forces which occur in 
at least one spatial direction while the writing 
20 movement of the pen is being carried out are recorded 
by pressure sensors, and correspondingly, at least one- 
dimensional, writing pressure data is produced. 

In one preferred embodiment of the method according to 
25 the invention, the character, the word or the word 
sequence is determined from the sequence of selected 
reference feature vectors, from the position data and 
from the writing pressure data by means of statistical, 
connectionistic and knowledge-based methods. 

30 

In one particularly preferred embodiment of the method 
according to the invention, the determined character is 
fed back optically and/or acoustically to the person 
producing it, in order to check it. 

35 

The generated personal-specific reference . data is 
preferably evaluated in order to determine 
psychological and/or physiological features of the 
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person . 

The biometric writing system according to the invention 
can be used widely. 

5 

In one preferred embodiment, the biometric writing 
system is used as a computer input device. 

In an alternative application, the writing system 
10 according to the invention is used as a speech input 
device, in particular as a dictation machine. 

The biometric writing system according to the invention 
is preferably used for identification of motor-neurone 
15 movement disturbances of a person. 

The biometric writing system according to the invention 
is furthermore preferably used as a therapy system for 
treating motor-neurone movement disturbances of person. 

20 

The biometric writing system according to the invention 
is furthermore and preferably used as a graphological 
system for determination of psychological/physiological 
features of a person. 

25 

The biometric writing system according to the invention 
is preferably used as a training system for learning to 
write . 

30 The biometric writing system according to the invention 
is preferably used as a training system for learning a 
language . 

Furthermore, preferred embodiments of the biometric 
35 writing system according to the invention and of the 
method according to the invention for personal 
identification, or personal verification, for 
generation of biometric reference data and for 
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handwriting and handwritten-sketch identification will 
be described with reference to the attached figures in 
order to explain features which are significant to the 
invention. 

5 

In the figures: 

Figure 1 shows a schematic figure in order to 

illustrate one preferred embodiment of 
10 the biometric writing system according 

to the invention; 

Figure 2 shows a section view through a pen 

housing of the biometric writing system 
15 according to the invention; 

Figure 3a-3e show a section view through pen housings 
according to preferred embodiments of 
the biometric writing system according 
20 to the invention; 

Figure 4 shows a flowchart in order to explain 

handwriting identification by means of 
the biometric writing system according 
25 to the invention; 

Figure 5 shows a sketch in order to explain 

visual and/or acoustic feedback to the 
writer; 

30 

Figure 6 shows acoustically recorded writing 

noise for a different writing speed on 
cardboard and on writing paper; 

35 Figure 7 shows an acoustically recorded sound 

time signal when writing the letter "a" 
three times; 
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Figure 8 shows an acoustically recorded sound 

time signal when writing the letter "b" 
three times; 



5 Figure 9 shows frequency signals when writing the 

letter "a M and "b" in the form of a time 
profile; 

Figure 10 shows an amplitude frequency signal when 

10 writing an associated letter in the "a"; 

Figure 11 shows the dynamic behaviour of a number 

of frequency lines when writing the 
letter "a"; 

15 

Figure 12 shows the dynamic behaviour of a number 

of frequency lines when writing the 
letter "b"; 



20 Figure 1 shows a schematic block diagram of the 
biometric writing system 1 according to the invention. 
The biometric writing system 1 according to the 
invention has a pen 2 which is integrated in a pen 
housing 3. The pen 2 is used to carry out hand-guided 

25 movements on a substrate 4. The substrate 4 may be any 
desired substrate with any desired surface. The 
substrate 4 is preferably paper. The writing substrate 
4 is preferably a fixed substrate, which has a specific 
pronounced surface roughness and hardness. 

30 

The pen 2 preferably contains an interchangeable pen 
refill, which can be extended from the pen 2 by means 
of a mechanical switching device, which is not 
illustrated. The pen housing 3 is used to carry out 
35 hand-guided movements on the substrate 4. In this case, 
in order to give the pen 2 and its pen refill a normal 
writing feeling, a writing liquid is emitted onto the 
substrate 4 in order to provide visual feedback for the 
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writer. When the pen 2 is placed on the substrate and 
it is moved on the surface, writing noises are 
produced. At least one microphone 5 for acoustically 
recording of sound signals is provided in the pen 
5 housing 3 in the writing system 1 according to the 
invention. The microphone 5 records the soupd signals 
which are produced by the hand-guided movements, that 
is to say the writing noise which is produced by the 
writing movement. The acoustic writing noises which are 

10 produced by the friction of the pen 2 on the substrate 
4 during the hand-guided writing movement are 
transmitted on the one hand as a structure-borne sound 
signal via the pen 2 and on the other hand as an 
airborne sound signal via the surrounding air to the 

15 microphone 5. In this case, the microphone 5 is 
mechanically coupled to the pen 2, in order to transmit 
the structure-borne sound signal. In the preferred 
embodiment shown in Figure 1, the microphone 5 is 
mechanically coupled to a sound body 6, which is 

20 connected to the pen 2, in order to transmit the 
structure-borne sound signal. The sound body 6 is 
preferably in the form of a resonator for specific 
natural frequencies . 

25 As is illustrated in Figure 1, the microphone 5 is 
located in an airborne sound chamber 7 which is 
provided in the pen housing 3. The airborne sound 
chamber 7 is preferably in the form of a resonator for 
specific natural frequencies. The microphone 5 is 

30 surrounded by sound insulation 8a, 8b, which is 
provided in order to attenuate surrounding noises and 
passes only sound signals from the sound body 6 and the 
airborne sound chamber 7. The airborne sound chamber 7 
can be coupled to the surrounding air via a housing 

35 opening which is provided in the pen housing 3. In this 
case, the housing opening can preferably be closed by 
means of a mechanical closure device 9, in order to 
suppress external noises. When the housing opening is 
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open, the microphone 5 records the internal and 
external writing noise which is caused by the hand- 
guided writing movement, as a structure-borne and 
airborne sound signal, as well as a speech signal which 
5 originates from a person. The microphone 5 converts the 
recorded acoustic sound signals to an electrical sound 
signal. The electrical sound signal is converted by an 
analogue/digital converter to sound signal data for 
digital data processing. The digital signal data is 

10 supplied via a signal line 10 to a data processing unit 
11 for further digital processing. The writing noise 
which is produced by writing and the speech signal 
which may additionally be received from the person are 
recorded either simultaneously or successively by the 

15 microphone 5, and are converted to corresponding sound 
signal data. The sound signal data which is supplied to 
the data processing unit 11 is preferably stored in a 
memory device 12 which is provided for this purpose and 
is connected to the data processing unit 11 via lines 

20 13. 

In addition to the microphone 5 as an acoustic sound 
sensor, the biometric writing system 1 according to the 
invention and as illustrated in Figure 1 preferably 

25 additionally has an optical sensor device 14, which 
records position data of the hand-guided pen movement 
via image signals from the surface of the writing 
substrate 4. The optical sensor device is likewise 
connected to the data processing unit 11 via a signal 

30 line 15. 

The microphone 5 may, for example, be an electret 
microphone, a piezoelectric microphone, a 

piezoresistive microphone or a capacitive microphone. 
35 The optical sensor device 14 preferably contains 
imaging optics for imaging the substrate surface, and a 
converter device for conversion of the optical imaging 
signal to an electrical signal. The electrical signal 
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is converted by an analogue/digital converter to 
position data or movement data, and is emitted via the 
signal lines 15 to the data processing unit 11. The 
imaging optics of the optical sensor device 14 are 
5 preferably in the form of optical lenses and/or glass 
fibres. The converter device may be a CCD camera or a 
photodiode array. A diode light source is preferably 
additionally provided, in order to illuminate the 
substrate surface. Furthermore, a laser diode is 

10 provided in order to illuminate an optical grating is 
integrated in the pen. The optical sensor device 14 
records the movements carried out by the pen 2 by 
comparison of the image sequences, which are recorded 
in the time offset and show the surface of the 

15 substrate. As an alternative to this, the optical 
sensor device 14 may also be designed in such a way 
that it records the movements which are carried out by 
the pen 2 by comparisons of the image sequences, which 
are recorded with the time offsets, of the interference 

20 of the laser light which is reflected on the grating 
and on the substrate surface. 

When in the extended state onto the substrate 4, the 
pen refill emits a writing liquid while the hand-guided 
25 movements are being carried out, which writing liquid 
increases the structuring of the surface so that 
optical recording of the hand-guided movements by means 
of the optical sensor device 14 is made easier. 

30 Furthermore, the ink written image on the substrate 
provides the writer with a natural writing style. 

The data processing unit 11 is, as is illustrated in 
Figure 1, integrated in the pen housing 3. In an 
35 alternative embodiment, the data processing unit may 
also be integrated in an external receiving unit. 
Examples of a receiving unit such as this may include a 
computer, a microcontroller, a portable telephone, a 
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credit card reader, a fax machine or a printer. 

The sound signal data which is recorded by the 
microphone 5 and the movement data which is recorded by 
5 the optical sensor device 14 are transmitted, if the 
data processing unit is located externally in a local 
computer, via a data transmission path 18 from the pen 
housing 3 to this local computer 20. As can be seen 
from Figure 1, the data processing unit 11 is connected 

10 via a signal line 16 to a downstream scrambling unit 
17. The scrambling unit 17 is used for scrambling the 
sound signal data, which is emitted in the data 
processing unit 11 within the pen housing 1, and the 
recorded optical movement data. Scrambled data is 

15 transmitted via the data transmission path 18, which 
uses wires or is wire-free, to a data processing unit 
19 within the local computer 20. In addition to the 
data processing unit 19, the local computer 20 has a 
memory 21 and a display 22, which are connected via 

20 respective signal lines 23, 24 to the data processing 
unit 19. The data processing unit 19 and the local, 
computer 20 is also connected via lines 25 to a reader 
unit 26. Furthermore, the data processing unit 19 in 
the local computer 20 controls an actuator 28, for 

25 example a safety or security door, via control lines 
27. The data processing unit 19 is connected via a line 
29 to a data network 30, to which an external database 
31 is connected. The data network 30 is preferably the 
Internet. 

30 

The biometric writing system 1 which is illustrated in 
Figure 1 contains an acoustic sensor device 5 and an 
optical sensor device 14. In one preferred embodiment, 
the biometric writing system 1 according to the 
35 invention additionally has a pressure sensor device, 
which records the static and dynamic pressure of the 
hand-guided pen which has been placed on the substrate 
4, and emits corresponding writing pressure data to the 
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data processing unit 11. The recorded sound signal 
data, the optical movement data and the recorded 
multidimensional pressure data are transmitted from the 
data processing unit 11, after appropriate scrambling 
5 by the scrambling unit 17, via the data transmission 
path 18 from the pen housing 3 to the data processing 
unit 19 which is integrated in the local computer 20. 
The data processing unit, which is located inside or 
outside the pen housing 3, uses the recorded sound 

10 signal data, the optical movement data and the 
mechanical writing pressure data to calculate biometric 
reference data, which is stored in a reference data 
memory, for example in the memory 12 or in the memory 
21. In alternative embodiments, the reference data 

15 memory may be a memory inside an identification card or 
a credit card, or an authorization magnetic card or a 
memory unit for any desired computer. The written words 
and characters whose writing noises are recorded 
acoustically by the microphone 5 and are emitted as 

20 sound signal data are, in particular, written pin 
codes, passwords, names or handwritten texts. The 
characters which are written by the person may at the 
same time be spoken by the person and may likewise be 
recorded by the microphone 5, as a speech signal. 

25 

The reading unit 2 6 which is provided in the local 
computer 20 is used, for example, for reading a 
portable memory medium for biometric reference data. 
Calculated, current biometric data can be compared in 

30 the data processing unit 31 with stored biometric 
reference data, for verification and identification. If 
the current biometric data largely matches the stored 
reference data, the data processing unit 11 or 19 
produces an identification and/or verification 

35 indication signal. In this case, once the 
identification and/or verification indication signal 
has been produced, an actuator, for example a security 
or safety door, is operated on the data processing unit 
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If the data processing unit 11 or 19 finds that the 
current biometric data completely matches the stored 
5 biometric reference data, the current biometric 
reference data is identified by the data processing 
units 11, 19 as being a stolen copy of the stored 
reference data, and an appropriate warning signal is 
produced . 

10 

The stored personal-specific biometric reference data 
is first of all generated by the biometric writing 
system 1 according to the invention, for subsequent 
comparison. The personal-specific biometric reference 

15 data is generated as follows. First of all, the hand- 
guided writing movements which are carried out by the 
pen 2 on the substrate while a person is writing a 
character, a word or a word sequence are recorded 
acoustically, that is to say the writing noise is 

20 recorded acoustically by the microphone 5. After 
analogue/digital conversion, the microphone 5 generates 
sound signal data, which is emitted to the data 
processing unit 11. The sound signal data is stored as 
a digital sound time signal, and an associated 

25 frequency spectrum is calculated from this stored time 
signal, by means of Fourier transformation. First 
biometric reference data is determined from the sound 
intensity of the stored digital sound times signal, by 
means of feature extraction. Second biometric reference 

30 data is determined from the calculated frequency 
spectrum of the sound time signal by means of feature 
extraction. These method steps are preferably carried 
out more than once, and the respectively determined 
reference data is evaluated statistically, with the 

35 evaluated data being stored in a scrambled form of 
personal-specific reference data. The statistically 
evaluated, personal-specific reference data is 
preferably stored in a microchip in an identification 
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card, a microchip in a credit card, in an authorization 
magnetic card or in a memory unit for a computer or 
writing system. In addition to the writing noise, the 
microphone 5 can additionally acoustically record a 
5 speech signal which originates from the person, and can 
emit corresponding sound signal data to the data 
processing unit 11. Preferably and additionally, the 
optical sensor device 14 optically records the hand- 
guided writing movements and produces corresponding 

10 writing movement data, which are emitted to the data 
processing unit 11. Furthermore and preferably, the 
writing forces which occur while the writing movement 
is being carried out by the pen 2 are recorded via 
pressure sensors which are not illustrated, and at 

15 least one-dimensional writing pressure data is emitted 
in a corresponding form to the data processing unit 11. 
The data processing unit 11 determines the biometric 
reference data from the sound intensity of the digital 
time sound signal by means of feature extraction, from 

20 the calculated frequency spectrum of the sound time 
signal by means of feature extraction, and from the 
optical movement data and writing pressure data which 
are produced while writing and speaking a word, by 
means of feature extraction. Data reduction of the 

25 static and dynamic writing movement data, of the 
writing pressure data, of the optical movement data and 
of the sound signal data is then preferably carried out 
by means of cluster formation. Feature parameters are 
then extracted from the data-reduced cluster data. 

30 

Figure 2 shows one preferred embodiment of a writing 
pen 2 for the biometric writing system 1 according to 
the invention. The pen housing 3 which is illustrated 
in Figure 2 is, for example, a conventional ball-point 
35 pen housing. The pen housing 3 contains a pen 2 with an 
integrated pen refill. The pen 2 is mechanically 
connected to a sound body 6 for transmission of 
structure-borne sound signals. The microphone 5 detects 
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the writing noise via the pen 2 and the sound body 6. 
In an alternative embodiment, the pen 2 may itself be 
used as a sound body. In addition to the writing noise, 
the microphone 5 can acoustically record a speech 
5 signal which originates from the person, when the 
airborne sound chamber 7 or the acoustic window is 
connected to the surrounding area by opening a closure 
device. The microphone 5 can record the writing noise 
and the speech signal simultaneously or successively. 

10 In the preferred embodiment illustrated in Figure 2, 
the microphone 5 is an electret microphone. In the 
embodiment illustrated in Figure 2, a loudspeaker 32 is 
additionally provided in the pen housing 3. The 
loudspeaker 32 is used to reproduce recorded microphone 

15 signals and, furthermore, the loudspeaker 32 can be 
used to reproduce stored biometric reference data and 
to reproduce spoken information. The loudspeaker 32 is 
preferably a miniaturized commercially available 
loudspeaker . 

20 

An acoustic writing signal or a writing noise is 
produced during writing with or movement of the pen 2 
on the substrate 4 . The pen 2 may be in the form of a 
ball-point pen refill, an ink knib or a wad with a 

25 defined tip. The substrate 4 is composed, for example, 
of paper or of a surface with a natural or additionally 
pronounced surface roughness and hardness. The surface 
roughness of the substrate 4 leads, while writing with 
the pen 2, to statistically stimulated, forced 

30 oscillations of the substrate and of the pen, that is 
to say statistical writing scratching noises and 
scrapping noises are produced from this . Depending on 
the design of the writing appliance and of the 
substrates, airborne sound loads and/or structure-borne 

35 sound loads are produced, from this and are transmitted 
to the microphone 5 within the pen housing 3. The pen 2 
together with the housing 3 in the form of a sound 
body, as well as the airborne sound chamber 7 in the 
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pen housing 3, can be designed optimally on the basis 
of acoustic laws. The sound body 6 and the airborne 
sound area 7 are preferably defined as resonators at 
specific frequencies. In this case, frequencies are 
5 preferably used which have the greatest dynamic range 
in terms of amplitude changes while writing. 

While writing with the pen 2, the microphone 5 emits a 
sound time signal which is significantly above the 

10 noise level and whose sound intensity depends on the 
writing speed, while a very small proportion of it is 
dependent on the different roughness of the writing 
substrate and on the different mean contact pressure of 
the pen refill 2. Sound intensity of the recorded sound 

15 time signal increases with the writing speed. If the 
writing speed is the same, the intensity differences 
between different writing paper surfaces are 
negligible. The frequency spectrum of the recorded time 
signal comprises a continuous noise spectrum, with 

20 significant characteristic frequency lines superimposed 
on it. The frequency lines occur mainly at frequencies 
below 2 kHz. The amplitudes of these frequency lines 
increase significantly with the writing speed, at the 
same frequency. The amplitudes of the frequency lines 

25 increase to a minor extent as the surface roughness of 
the writing substrate 4 increases. In this case, the 
frequencies remain unchanged. For the same writing 
speed, the amplitudes are virtually independent of the 
writing substrate 4 that is used. The microphone 5, 

30 which attenuates airborne sound, is insensitive to 
sound events in the surrounding area. If the pen 
housing 3 is completely closed, the airborne sound 
chamber 7 has no connection to the surrounding air. If 
the airborne sound chamber 7 is closed, environmental 

35 noises, in particular spoken words or motor noises, 
have no influence on the recorded sound time signal 
while writing. In this case, the microphone 5 recorded 
only the structure-borne sound that is produced via the 

TRIl\605845vl 



- 31 - 

pen refill. 

The biometric features of the person's writing dynamics 
are detected from the changeover time in the acoustic 
5 microphone signals and/or the amplitudes of selected 
spectral lines . Since only the dynamics of the recorded 
sound signals are evaluated for the generation of 
biometric reference data, the influence of different 
roughnesses of the substrate 4 and the influence of 

10 manufacturing scatters in the production of the writing 
appliance are negligible. Scatters in the writing 
dynamics are thus a function of the person, and are 
specific to that person, and can thus be evaluated for 
identification and verification of that person. The 

15 sound time signal which is recorded acoustically by the 
microphone 5 is, in contrast to writing pressure 
signals which reflect the pressure of the pen refill in 
different directions, independent of how the pen 2 is 
held on the substrate 4. The sound signal which is 

20 produced by the microphone 5 does not change if the pen 
housing 3 is held such that it is rotated differently 
about the longitudinal axis. The geometry, the 
material, the layer structure, the attenuation and the 
acoustic coupling between the sound source and the 

25 microphone 5 of the writing appliance as illustrated in 
Figure 2 can in this case be designed so as to allow 
the generation of optimum biometric features, which 
acoustically cause little disturbance for 

authentication methods. An acoustic window or an 

30 opening, which the person can close by pushing a button 
in order to record speech, is provided in the pen 
housing 3. A relatively small opening in the pen 
housing 3 is sufficient for this purpose, in order to 
also record the airborne sound via the microphone 5, 

35 while speaking. The person closes the housing window in 
order to record the pure acoustic writing signal or 
writing noise. 
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Figure 3a shows a further preferred embodiment of the 
biometric acoustic writing system 1 according to the 
invention. In the particularly preferred embodiment 
that is illustrated in Figure 3a, further sensor 
5 devices are additionally provided in the pen housing 
3a, in addition to the microphone 5 for acoustic 
recording of writing noises. In addition, the biometric 
pen which is illustrated in Figure 3 contains an 
optical sensor device 33. The optical sensor device 33 

10 records the position data of the hand-guided pen 
movement via image signals from the surface of the 
writing substrate 4 . The optical sensor device 33 
contains imaging optics for imaging of the substrate 
surface, and a converter device for conversion of the 

15 optical imaging signal to an electrical signal. 

The imaging optics are in the form of an optical lens 
33a. The lens optics may be a glass-fibre bundle or a 
GRIN lenses. The converter device 33c of the optical 

20 sensor device 33 is, for example, a CCD camera with an 
integrated microprocessor. Alternatively, a photodiode 
array may be used. A diode light source is preferably 
additionally provided in order to illuminate the 
substrate surface. In one alternative embodiment, the 

25 laser diode is additionally used to illuminate an 
optical grating which is integrated in the pen. 

The optical sensor device 33 records the movements 
which are carried out by the pen 2, by comparison of 

30 the image sequences, which are recorded with the time 
offset, of the substrate surface. In one alternative 
embodiment, the optical sensor device records the 
movements which are carried out by the pen 2 by 
comparison of the image sequences, which are recorded 

35 with the time offset, of the interference of the laser 
light which is reflected on the grating and on the 
substrate surface. In the extended state on the 
substrate 4, the pen refill 2 emits a writing liquid 

TRIl\605845vl 



- 33 - 

while carrying out the hand-guided movements, which 
increases the structuring of the substrate surface. 
This makes it easier to optically record the hand- 
guided movement by means of the optical sensor device 
5 33, and the ink script provides the person with 
feedback, this making it easier to write naturally. 

The first preferred embodiment of the biometric writing 
system 1 according to the invention, as illustrated in 

10 Figure 3a, also contains a pressure sensor system 34 
for recording the writing pressure, as well, in 
addition to the microphone 5 for recording the acoustic 
writing noise, and the optical sensor device 33 for 
generation of position and movement data. The pressure 

15 sensor system 34 is, for example, in the form of strain 
gauges DMS or force-resistive resistances FSR, as well 
as piezo sensors 34a, 34b, 34c, 34d, whose sensors 
record the static and dynamic pressure of the hand- 
guided pen 2 which has been placed on the substrate 4 . 

20 The sound signal data which is emitted by the 
microphone 5, the movement data which is emitted by the 
optical sensor device 33 and the recorded multi- 
dimensional writing pressure data are supplied to an 
integrated data processing units 11a, lib within the 

25 pen housing 3, and are processed to form biometric 
data. The biometric data that is determined is 
preferably temporarily stored in a data memory 12 
within the pen housing 3. The currently determined 
biometric data is then preferably scrambled by means of 

30 a scrambling unit 17, and is transmitted via a data 
transmission path 18 to a remote computer 20. 

In the embodiment illustrated in Figure 3a, the 
acoustic window 7 can be switched between voice 
35 recording and writing noise recording by moving sound 
insulation 8b within the pen housing 3. For example, in 
a loud environment, the interfering external noise can 
be suppressed by closing the acoustic window 7. In 
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order to increase the identification confidence, the 
person can open the acoustic window and, for example, 
can additionally speak a written password at the same 
time, or with a time offset. 

5 

Figure 3b shows a second preferred embodiment of the 
biometric writing system 1 according to the invention. 
The data processing unit 11 is supplied with electrical 
power from a rechargeable battery 37. A pressure sensor 
10 device 34b is provided at the upper end of the pen 
refill 2. Furthermore, a knib 38 is provided. The 
pressure sensor 35b is preferably a piezo pressure 
sensor. 

15 Figure 3c shows a third preferred embodiment of the 
writing system 1 according to the invention. The 
writing system 1 is rotationally symmetrical and 
comprises a pen refill 2, a microphone 5 with a holder, 
sound insulation 8 and a piezo pressure sensor 34. 

20 

Figure 3d shows a fourth preferred embodiment of the 
writing system 1 according to the invention. A 
fingerprint sensor 39 is provided on the pen housing 3. 
The fingerprint sensor 39 records biometric data 

25 relating to the person who is writing, via body 
features. The fingerprint sensor is preferably a 
capacitive fingerprint sensor. The pen 2 is mounted in 
the pen housing 3 by means of a spring steel folder 42. 
The pen 2 has two microphones 5a, 5b. A thinner 

30 microphone 5a is provided within a body 49 (which is 
provided in the pen housing 3) and is located in the 
vicinity of a face 41. The face 41 is used as an 
acoustic oscillator, and is connected to the pen refill 
2. An external microphone 5b is used to record the 

35 environmental sound. The embodiment of the writing 
system 1 illustrated in Figure 3d also has an 
inclination sensor 40. The inclination sensor 40 
records the inclination of the pen 2 during writing, 
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thus recording the motor movement of the finger which 
is . guiding the pen. The data processing unit 11 is 
supplied with electrical power from a rechargeable 
battery 37. A pressure sensor 34b is also located at 
5 the upper end of the pen 2. Further pressure sensors 
34a, 34b, 34c are provided on the spring steel holder 
42. The spring steel holder 42 is used as a pen refill 
holder and as a deformation device for the strain 
gauges 34a, 34b, 34c, which are preferably fitted to 
10 the spring steel holder 42. 

Figure 3e shows a fifth preferred embodiment of the 
biometric writing system 1 according to the invention. 
The writing system 1 has an optical sensor device 48, 

15 which records the static and dynamic pressure and the 
oscillation of the hand-guided pen 2, which has been 
placed on the substrate 4, in three special directions 
at the same time. This optical sensor device 48 
contains photodetectors for recording the movement of a 

20 diode light source 44. The diode light source 44 is 
firmly connected to the pen 2 . The diode light source 
44 carries out the deflection and oscillation of the 
pen 2, with the light beams which are emitted from the 
diode light source 44 being emitted as a light source 

25 system to the photodetectors of the optical sensor 
device 40. The light beams which are emitted from the 
diode light source 44 pass to a beam splitter 47, and 
from there to a lens 46, which preferably has a 
partially reflective lower face. The optical sensor 

30 device 40 furthermore contains a four-quadrant 
photodetector 50 as well as a single photodetector 45. 
The four-quadrant photodetector 50 is fitted behind the 
lens 46. The four-quadrant photodetector 50 records the 
deflection and the oscillation of the light source 44 

35 in mutually orthogonal X, Y directions. The single 
photodetector 45 detects the deflection and oscillation 
of the light source 44 in the direction Z, which is at 
right angles to the X and Y directions. The dynamics of 
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the writing pressure can be determined via the 
deflection, and the dynamics of the writing speed can 
be determined by the oscillation, in three special 
directions from the signals from the four-quadrant 
5 photodetector 50 and from the single photodetector 45. 
The signals from the photodetectors 45, 50 are supplied 
to the data processing unit 11, where they are 
evaluated. In the embodiment illustrated in Figure 3d, 
the pen 2 is held by means of a pen refill holder 43. 
10 Furthermore, a knib 38 is provided. The image of the 
optical sensor device 40 comprises the beam splitter 
47, the optically partially reflective lens 46 and the 
shutter . 

The biometric acoustic writing system 1 illustrated in 
Figures 1 to 3 can be used in a versatile manner. The 
biometric writing system 1 can be used as a computer 
input device. Furthermore, the biometric writing system 
1 is suitable for use as a speech input device, in 
particular as a dictation machine. 

A further application possibility is identification and 
verification of a person who is writing by comparison 
of currently obtained biometric data with stored 
25 personal biometric reference data. When a person is 
being verified, a check is carried out in the process 
to determine whether, for example, the signature given 
actually originates from the correct person. For this 
purpose, the currently generated biometric data is 
30 compared with stored reference data for this person. 
One precondition for this is that the person-specific 
biometric reference data for this person must first of 
all be generated. For this purpose, hand-guided writing 
movements which are carried out by this person under 
35 supervision are first of all recorded acoustically. The 
person carrying this out signs, with his signature, for 
example, under supervision, and the microphone 5 within 
the writing appliance records sound signals of the 
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writing noise that occurs during this process. The 
recorded writing noises are emitted as sound signal 
data to the data processing unit 11. The sound signal 
data is temporarily stored, as digital sound time 
5 signals, in a memory 12. The data processing unit 11 
then calculates the associated frequency spectrum, for 
example by means of Fourier transformation. First 
biometric reference data is determined from the stored 
digital sound time signal, by means of feature 

10 extraction. Second biometric reference data is 
determined from the calculated and stored frequency 
spectrum of the sound time signal, likewise by means of 
feature extraction. These method steps can preferably 
also be carried out more than once, and the 

15 respectively determined biometric reference data is 
evaluated statistically. The evaluated data is then 
scrambled and stored as personal-specific reference 
data. The biometric reference data which is generated 
can then be used for personal verification and personal 

20 identification in a large number of applications. The 
personal-specific reference data which is generated is, 
for example, stored in a microchip in an identification 
card, in a microchip in a credit card, in an 
authorization magnetic card or in any desired memory 

25 unit for a computer. During the generation of the 
biometric reference data, a speech signal which 
originates from the person is preferably additionally 
recorded acoustically, and corresponding sound signal 
data is produced. For example, the relevant person can 

30 produce a signature at a bank or in the presence of 
some authority, under supervision, and can speak his or 
her own name at the same time or subsequently. In order 
to further improve security, the writing forces which 
occur while producing the signature can additionally be 

35 recorded and evaluated by means of the pressure sensor 
system. A further improvement in security is obtained 
by additional optical recording of the hand-guided 
writing movement, that is to say optical recording of 
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the position data of the signature that is produced. 
The pen which is illustrated in Figures 3a-3e uses a 
sensor system to record a large number of personal- 
specific features, in particular the written image of 
5 the signature that is produced, the static and dynamic 
writing pressure which occurs while the writing the 
signature, the writing noise which is produced while 
writing the signature and, additionally, the word which 
is spoken by the person producing the writing, for 

10 example his or her name and a password. All of this 
data is processed to form biometric reference data, and 
is stored. The identification confidence and the 
confidence against falsification are in this case 
already high since widely differing personal-specific 

15 features are recorded and evaluated by the sensor 
system. For personal verification, the current 
biometric data generated by the writing system is 
compared with the previously generated and stored 
biometric reference data for that person in order to 

20 confirm whether the current biometric data largely 
matches the stored biometric reference data for that 
person. However, if there is a one hundred percent 
match, there is assumed to be a stolen copy of the 
stored reference data, and the biometric writing system 

25 1 according to the invention emits an appropriate 
warning signal. 

For personal identification, the current biometric data 
is compared with stored biometric data for a large 
30 number of people, and that person is selected whose 
biometric reference data best matches the current 
biometric data. 

In addition to personal verification and personal 
35 identification, the biometric writing system according 
to the invention can also additionally be used for 
handwriting identification and /or handwritten-s ketched 
identification. Figure 4 shows, schematically, the 
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steps for handwriting identification. The writing 
noises which occur while the pen 2 is writing on the 
substrate 3 while writing or sketching a character, a 
word or a word sequence are recorded acoustically by 
5 the microphone 5, and are converted to corresponding 
sound signal data. Once the acoustic signal processing 
is carried out, this is followed by coding. The sound 
signal data which is produced is temporarily stored as 
a digital sound time signal, and the associated 
10 frequency spectrum is then calculated from the sound 
signal data by means of Fourier transformation. 

In a further step, first current acoustic data is 
determined from the sound intensity of the digital 
sound time signal by means of feature extraction, for 
data reduction. Current second acoustic data is 
determined from the calculated frequency spectrum of 
the sound time signal, likewise by means of feature 
extraction. This step as well is also primarily used 
for data reduction. Finally, a sequence of feature 
vectors is calculated from the acoustic data, and these 
calculated feature vectors are compared with stored 
personal-specific reference feature vectors in a 
pattern comparison process. A sequence of reference 
feature vectors which differ least from the respective 
feature vectors is then selected. The written 
character, the written word or the written word 
sequence is determined from the sequence of selected 
reference feature vectors by means of statistical, 
connectionistic and knowledge-based methods. 

The statistical methods are, for example, hidden Markow 
models for identification of a relatively large 
vocabulary. The connectionistic methods primarily make 
35 use of neural network models for identification of a 
limited vocabulary. The knowledge-based methods are, 
for example, fuzzy logic models. The reference feature 
vectors are system-specific for the position of the 
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writing system and are produced and stored on a 
personal-specific basis by repeatedly writing a 
training text by hand, before the use of the system. 
The reference feature vectors for that person are 
5 updated adaptively and automatically via newly written 
characters and words over lengthy periods of use. All 
the methods are based on acoustic models of characters, 
letters, syllables and words in the vocabulary of 
spoken speech. The spoken speech vocabulary is formed 

10 by spoken elements, that is to say brief spoken units 
(allophones) in the written noise. Each word in the 
vocabulary is stored as a sequence of allophones. The 
acoustic models use the spoken vocabulary to determine 
a number of word candidates which are most probably 

15 associated with the acoustic symbol sequence. The 
probability for each word is in this case determined 
via the HMM (Hidden Markow Models) or the neural fuzzy 
systems. The most probable word candidate is determined 
via a threshold value decision and via a word sequence 

20 statistics model or speech model. The character which 
is determined is then emitted, by way of example, as 
formatted text or as handwritten text. In order to 
improve the identification confidence, the written 
character or word is preferably additionally ' spoken 

25 like the person, and is recorded acoustically by the 
microphone. In addition, the position data for the 
hand-guided writing movement is recorded optically. A 
further increase in confidence is achieved by recording 
the writing pressure data which, occurs while writing 

30 the word. 

The personal-specific reference data which is generated 
by the writing system 1 according to the invention can 
be evaluated in order to determine psychological and/or 
35 physiological features of that person. In particular, 
the occurrence of motor-neurone movement disturbances 
of a person can be identified by means of the biometric 
writing system 1 according to the invention. The 
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biometric writing system according to the invention is 
thus suitable for diagnosis of psychological and/or 
physiological features, and for diagnosis of motor- 
neurone movement disturbances of that person. 

5 

Figure 5 shows one particularly preferred embodiment of 
the biometric writing system 1 according to the 
invention with opto-acoustic feedback for the person 
who is producing the writing. The biometric data which 

10 is obtained from the pen 3 is transmitted to a local 
computer 20, which provides opto-acoustic feedback via 
lines 35a, 35b to the person who is producing the 
writing, via a screen 36a and a loudspeaker 36b. In 
this configuration, the biometric writing system 1 

15 according to the invention is suitable for use as a 
system for therapy for people with movement 
disturbances. The opto-acoustic feedback allows the 
person who may have a motor-neurone movement 
disturbance to learn to write again. 

20 

A further interesting field of application is the use 
of the biometric writing system 1, as is illustrated in 
Figure 5, as a training system for learning to write. 
In particular, the biometric writing system 1 according 

25 to the invention is suitable for use as a training 
system for learning a language, in particular a foreign 
language. The person being taught may, for example, 
write a word on the substrate 4, and the loudspeaker 
which is integrated in the pen housing 3 reproduces the 

30 word in the correct spoken version of the foreign 
language. Predetermined numbers or characters which are 
produced on the substrate 4 can be copied by the person 
with the pen by means of a training program for 
students or people with coordination difficulties, for 

35 example stroke patients. By way of example, a student 
can copy a predetermined number for a character on the 
substrate 4, and receives optoacoustic feedback. 
Writing pressure recording allows documentation of the 
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learning progress for people with coordination 
difficulties. As is illustrated in Figure 5, it is 
possible to use the biometric writing system 1 
according to the invention for children to learn to 
5 write the alphabet in a simple manner. Furthermore, 
fabrication practices, information conveyance practices 
and imitation practices are possible. 

Figure 6 shows the behaviour of the writing system 1 
10 according to the invention when writing on different 
substrates 4. Figures 6a, 6b show the recorded time 
sound signal for different writing speeds on a 
cardboard surface and on a writing paper block. In this 
case, in the illustrated example, a line is drawn at 
15 increasing speed and the cardboard and on the writing 
paper block. Figures 6c, 6d show the associated 
frequency spectra . 

Figure 7 shows a time sound signal which is generated 
20 when the letter in the "a" is written three times, when 
the pen is being used to write on a writing block by 
means of the biometric writing system 1 according to 
the invention. As can be seen from Figure 7, the sound 
time signal can be reproduced very well, so that the 
25 letter "a" can be identified specifically. 

Figure 8 likewise shows a time sound signal which is 
produced when a letter "b" is written three times. The 
letter "b" also has a characteristic sound time signal 
30 profile. 

Figure 9a shows amplitude frequency signals in the time 
profile for the letter "a", and Figure 9b shows 
amplitude frequency signals in the time profile for 
35 writing the letter "b" . 

Figures 10a, 10b show, in principle, how the amplitude 
frequency lines in the time profile change dynamically 
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while writing a letter, for example a letter "a". The 
person who is producing the writing places the pen on 
the substrate (initial point AP) at the time tO, and 
carries out the writing movement. The spectrum which 
5 exists at the time tO changes dynamically in the time 
profile. Figure 10b shows the change in the spectrum 
from the time tO to the time tl . 

Figure 11 shows the dynamic response of a number of 
10 frequency lines while writing a letter "a". In the 
illustrated example, the best reproduction of the 
letter "a" occurs for the frequency line 
f = 86.133 Hertz. 

15 Figure 12 shows the dynamic response of an a number of 
frequency line while writing a letter "b" . The best 
reproduction of the letter M b" is provided by the 
frequency line f = 86.133 Hertz. 

20 The acoustic writing system 1 according to the 
invention can be designed in such a way that it records 
the frequency lines of the handwritten characters over 
as wide a dynamic range as possible with large 
amplitude values. 

25 

The biometric acoustic writing system 1 according to 
the invention has a large number of application 
possibilities. On the one hand, it can be used as a 
biometric authentication system for personal 

30 identification and verification. The writing system 1 
according to the invention is suitable for use in a 
computer input device, as a replacement for a computer 
mouse or keyboard commands, in particular for entering 
passwords. Furthermore, the biometric writing system 1 

35 according to the invention can be used for electronic 
voice and handwriting identification. A further 
extensive field of application relates to the field of 
biometric diagnosis and therapy. Finally, the biometric 
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writing system 1 according to the invention can be used 
for training purposes, for example for learning to 
write, for learning foreign languages or for correcting 
coordination difficulties. Important fields of use of 
5 the writing system according to the invention occur in 
the area of E-commerce, in particular in the Internet, 
for example in the case of banks, commercial 
organizations and for credit card matters. The 
biometric writing system 1 according to the invention 

10 can also be used as a biometric system for personal 
access and security checks in all security areas with a 
high degree of confidence against falsification and 
with low error rates. Further usage options exist for 
forensic purposes for' criminal investigations, for 

15 border protection or for the police. Furthermore, the 
biometric writing system 1 according to the invention 
can be used for communication between people and 
machines, particularly as a dictation machine, or for 
verbal and handwritten machine control. 

20 

The multifunctional biometric writing system 1 is also 
suitable for use as an input unit for a future virtual 
desktop, in which, by way of example, the tablet of a 
PC can be added to or replaced, via the pen, by means 
25 of the normal writing paper, as an input medium with 
the input capabilities according to the invention. 

The biometric writing system 1 according to the 
invention, as is illustrated in Figures 1 to 3, can be 
30 produced at low cost with relatively little technical 
complexity, thus opening up wide fields of use. 
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List of reference symbols 

1 Biometric acoustic writing system 

2 Pen 

5 3 Pen housing 

4 Substrate 
5a, 5b Microphones 

6 Sound body 

7 Airborne sound chamber 
^10 8, 8a, 8b Sound insulation 

9 Closure device 

10 Signal line 

11 Date processing unit 

12 Memory unit 
15 13 Lines 

14 Optical sensor device 

15 Signal lines 

16 Signal lines 

17 Scrambling unit 

20 18 Data transmission path 

19 Data processing unit 

2 0 Local computer 

2 1 Memory 

22 Display device 
25 23 Line 

24 Line 

25 Line 

2 6 Reading unit 

2 7 Control line 
3 0 .28 Actuator 

29 Line 

3 0 Data network 
31 Database 

3 2 Loudspeaker 

3 5 3 3 Second optical sensor device 

34 Pressure sensor device 

3 5 Line 

36 Acoustic feedback devices 
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... 


37 


Rechargeable battery 


r 


38 


Spring 




39 


Fingerprint sensor 




40 


Inclination sensor 


5 


41 


Cord 




42 


Spring steel holder 




43 


Pen refill holder 




44 


LED 




45 


Photodetector 


10 


46 


Lens 




47 


Beam driver 




48 


First optical sensor 




49 


Body 




50 


Four-quadrant phot ode 



15 
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List of reference symbols 
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1 




Biometric acoustic writing s 


2 




Pen 






Pen housing 


/I 




bubs urate 


c: _s 


3D 


Microphones 


o 




Sound body 


1 




Airborne sound chamber 


Q 


Q 

oa , 


8b Sound insulation 






Closure device 


1 U 




Signal line 


1 1 




Date processing unit 


1 9 
1 z 




Memory unit 


1 "3 
1 O 




Lines 


1 /I 
1 4 




Optical sensor device 


1 

1 o 




Signal lines 


1 D 




Signal lines 


17 




Scrambling unit 


18 




Data transmission path 


1 9 




Data processing unit 


z u 




Local computer 


2 1 




Memory 


22 




Display device 


2 J 




Line 


2 4 




Line 


2 O 




Line 


9 £ 




Reading unit 


9 *7 
Z / 




Control line 


o o 
2 o 




Actuator 


2 y 




Line 






L/clL-ci IlcLWOlK 


31 




Database 


32 




Loudspeaker 


33 




Second optical sensor device 


34 




Pressure sensor device 


35 




Line 


36 




Acoustic feedback devices 
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37 Rechargeable battery 

38 Spring 

3 9 Fingerprint sensor 

40 Inclination sensor 

41 Cord 

42 Spring steel holder 

43 Pen refill holder 

44 LED 

45 Photodetector 

4 6 Lens 

47 Beam driver 

48 First optical sensor device 
4 9 Body 

50 Four-quadrant photodetector 



TRIl\605845vl 



